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Abstract 

Instructional written materials play important roles as teachers’ agents in effective teaching practices. 
Worksheets are one of the most frequently used materials. In this exploratory study, the relationships between 
worksheet usage and science achievement in 32 countries were examined through the use of TIMS S and PIRLS 
data and multiple regression analysis. Based on two dimensions, five types of relationships among science 
achievement, worksheet usage, and other related variables are identified. The first dimension is whether the 
status of significance in the association of worksheets used as a basis and science achievement changes before 
and after controlling four teacher and school variables: schools’ emphases on academic success, safety and 
orderliness of school, teachers’ confidence in teaching science, and instructional engagement of students. The 
second dimension is the interaction of worksheets as a basis and classes’ lack of readiness. The interaction 
between worksheets as a basis and reading achievement in science achievement is found to be not significantly 
different from zero in all participating countries. Four directions of further investigation are suggested based on 
the results. 

Key words: Science education. Worksheets, Elementary education, Secondary analysis 


Introduction 

Worksheets have been used in teaching practices for a long time. In modern time, worksheets have even become 
a driving force of curriculum in some countries (Lesley & Labbo, 2003; Martin, Mullis, Foy, & Stanco, 2012; 
Reid, 1984). Anderson et al. (1985) reported that in 1985, thousands of elementary students in the United States 
completed approximately 1,000 worksheets per person to acquire literacy in a school year. Teachers use 
worksheets for the purposes of supporting studying, promoting active learning, raising interest in learning 
science, and assessment. Many studies suggest that well-designed worksheets have had positive impacts on 
students’ learning achievement (Sasmaz-Oren & Ormanci, 2012). However, researchers observed that there 
were many inappropriately designed and misused worksheets that hindered learning (Lesley & Labbo, 2003). In 
this exploratory study, the relationships between worksheet usage and science achievement in 32 countries are 
examined. 


Worksheet and Achievement 

Worksheets can be useful in many ways in terms of academic achievement. For example, as supplements to 
textbooks, worksheets can be used to add information for particular classes. In addition, blanks in worksheets 
are invitations for students to fill in gaps; they are opportunities for knowledge construction. Well-designed 
questions in worksheets can draw students’ interest when paired with proper teaching methods. Furthermore, 
worksheets play a variety of functions in different contexts. McDowell and Waddling (1985) suggested that 
during laboratory investigations, properly designed worksheets can help teachers overcome the problems of time 
demanding and enable teachers to enhance students’ acquisition of knowledge and skills. Kisiel (2003) pointed 
that in activities while visiting museums, worksheets can function as advance organizers, helping students to 
organize their observations and knowledge in a confusing learning environment. Krombab and Harms (2008) 
concluded that worksheets are effective in helping students aged 11-15 to acquire knowledge such as 
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biodiversity in a natural history museum because they can structure the visit, keep students’ attention on certain 
objects, and form a basis for follow-up coursework. As an assessment tool, worksheets can be used by teachers 
to understand students’ previous knowledge, outcome of learning, and the process of learning; at the same time, 
they can be used to enable students to monitor the progress of their own learning. 

In order to ensure worksheet’s effectiveness, many studies focused on the design of worksheets (Campbell, 
1999; Hoener, Salend, & Kay, 1997; Sasmaz-Oren & Ormanci, 2012). The basis for successfully transferring 
message to students is layout. To enhance teachers’ abilities to design worksheets. Rotter (2006) proposed the 
layout principle COLA (contrast, orientation, lettering, and artwork). In worksheets, characteristics of questions 
are important factors. Calderhead et al. (2006) demonstrated that the arrangement of items with different levels 
of cognitive difficulty can affect students’ learning results. Formats of information, or scaffolds, in worksheets 
are also of concern to educators. For instance Wolf et al. (2010) devised three formats of critical thinking tools 
to promote conceptual understanding in physical geography and Ueckert and Gess-Newsome (2008) advocated 
for a “conceptual flow graphic” to make active learning possible. 

In addition to the positive impacts of worksheets on academic achievement, there are negative impacts. Lesley 
and Labbo (2003) argued that mass-produced worksheets are not helpful for achieving educational goals. 
Although the worksheets they observed were focused on literacy learning, their findings can still shed some 
light on worksheet usage for science learning. According to their remarks, the aspects of worksheet problems 
included the format of texts (e.g., that the print and the spaces allotted for students to write in are too small); 
reading demand (e.g., that the language of instruction was too complex and required teacher explanation); 
openness of questions, some of which offered only one correct way to respond and could not reward students for 
their natural curiosity; the challenge of tasks (e.g., that tasks were boring or designed for practicing skills 
repeatedly instead of making students learn new strategies or techniques), and the relationship between students’ 
interests and tasks. In addition to the quality issues, as students completing worksheets, their cognitive processes 
can also make worksheets invalid. Ueckert and Gess-Newsome (2008) noted that students use a word-matching 
strategy match words in questions with the corresponding sentences in the textbook, and this keeps them in a 
passive learning status. 

Worksheet, Reading Achievement, and Science Achievement 

Worksheets are a kind of written material, so reading demand may be a barrier to students with low reading 
abilities. Researchers suggested that teachers should use easier language to support students (Rix, 2006). For 
example, O’Leary (2011) designed a format of worksheets with low average reading difficulty. Questions in the 
beginning are carefully matched with low reading ability students and subsequent questions require increasing 
levels of literacy. The result showed that this kind of worksheets can improve student engagement and on-task 
behavior during independent worksheet activities. There are a number of factors contributing to reading 
difficulty, such as organization of materials, syntax, word length, sentence length, word frequency, typeface, and 
line spacing (Department of Education and Science, 2007; Meyer, 2003; O’Leary, 2011). If teachers carefully 
control these factors and use available readability formulas to reduce the reading demand, or offer oral 
explanations to words in worksheets, the association between worksheet usage and science achievements will be 
the same regardless of students’ reading achievements. 

The debate about the difficulty of language used in worksheets has been raised in literature (O’Leary, 2011). 
Some researchers insist that easy language negates worksheets’ abilities to challenge students and offer 
opportunities of language acquisition (Hayes, Wolfer, & Wolfe, 1996). In addition, although there are rules for 
controlling the reading demand, according to Reid’s (1984) study, only about 30% of worksheet writers checked 
readability despite knowing that it is the essential. Under this kind of condition, students with low reading 
abilities have problems completing worksheets and the associations between worksheet usage and science 
achievement therefore differ among students with different reading levels. 

Worksheet, Lack of Readiness, and Science Achievement 

According to Reid (1984), teachers tend to use worksheets with low-achievement classes. The reasons for this 
tendency may be twofold. One is that textbooks are designed for general students and need to be adapted. 
However, worksheets can offer relevant questions and motivate students, both of which are functions that were 
ranked as best performed by teachers surveyed in the Reid’s (1984) study. Secondly, as written material, 
worksheets are able to act as agents of teachers to lead students’ attentions and give students opportunities to 
work independently, so the students can work at their own paces and the teacher can have time to take care of 
those students who need more help (McDowell & Waddling, 1985). 
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If worksheets are properly designed and used, they can support students’ thought. However, many teachers are 
concerned by the “repetitive” nature of worksheets, the risks of student boredom, and a lack of the pedagogical 
knowledge and skills required to prevent children from thinking insufficiently before completing worksheets 
(Reid, 1984). These factors may cause failure in the application of worksheets in classes lacking readiness. 

Research Questions 

The purpose of this study is to explore the association between worksheet usage and science achievement in 
grade four students across the countries in the fifth cycle of the Trends in International Mathematics and Science 
Study (TIMSS 201 1) and the third cycle of the Progress in International Reading Literacy Study (PIRLS 201 1). 

To gain an overview of the association, the first question that arises is: “Is there difference in science 
achievement between situations in which worksheets are used as a basis for instruction and those in which they 
are used as supplements, and between those in which they are used as supplements and those in which they are 
not used?” 

To avoid the confounding effect of other teacher and school level variables, the four important variables, 
schools’ emphasis on academic success (EAS), safty and orderliness of school (SOS), teachers’ confidence in 
teaching science (CTS), and instructional engagement of students (lES), identified by Martin et al. (2012) are 
controlled. Accordingly, the second question is “After the four variables are controlled, is there difference in 
science achievement (SA) between those situations in which worksheets are used as a basis (WB) and those in 
which they are not used as a basis?” 

In considering the issues about reading achievement (RA) and classes’ lack of readiness (LR), the third and 
fourth questions examine “whether RA contributes to explanations of the variability in the relationship between 
WB and SA” and “whether LR contributes to explanations of the variablility in the relationship between 
worksheet as a basis (WB) and SA.” 


Method 

Sample 

This study is a secondary analysis research. Data were collected in TIMSS 2011 and PIRLS 2011. Thirty-four 
countries participated in both surveys (Martin & Mullis, 2013). The target population is grade four students in 
thirty-two countries and grade six students in two countries. In this study, the data from the thirty-two countries 
were analyzed. 

TIMSS and PIRLS use two-stage stratified cluster sampling design (Martin et al., 2012). In each county, schools 
were selected first and one or two classes were selected from those schools. The students in the selected classes 
composed the sample. The sample size of each country was ranged from 3,121 to 14,720. 

Dataset Preparation 

After scaling, the International Association for the Evaluation of Educational Achievement (lEA) released the 
TIMSS and PIRLS 2011 datasets in September 2013. In the datasets, there are reading, science, and 
mathematics achievement scores with five plausible values; responses of items in achievement tests and 
background questionnaires; and derived background variables, such as students’ confidence in learning science, 
teachers’ confidence in teaching science, and safety and orderliness of school (Eoy, 2013). 

In this study, there is one dependent variable, SA, and seven independent variables: WB, RA, LR, EAS, SOS, 
CTS, and lES. SA and RA are achievement scores. EAS, SOS, CTS, and lES are derived teacher and school 
variables. The above six variables can be directly retrieved from the datasets provided by lEA (Eoy, 2013). WB 
and LR are defined in this study as the following. 

Worksheet usage was one item in the teacher questionnaire (Martin et al., 2012). Teachers were asked when 
they taught science to the class how they used “workbooks or worksheets.” There were three options for them to 
choose from: “basis for instruction,” “supplement,” and “not used.” Because the percentage of teachers who did 
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not use workbooks or worksheets was small or even zero in some countries, the responses were recorded in two 
categories, “as a basis” and “not as a basis.” 

The variable of LR was constructed with the use of three items in the teacher questionnaire. In the questionnaire, 
teachers were asked to what extent “students lacking prerequisite knowledge or skills,” “disruptive students,” 
and “uninterested students” limited how they taught their classes. There were four options for them to choose 
from: “a lot,” “some,” “not at all,” and “not applicable.” Every item was recoded in two-level variable: “A lot or 
some (level 1)” and “not at all or not applicable (level 0).” The three indicators were treated as formative, and 
LR was the standardized sum of the three recoded variables. 

Analytical Strategy 

Multiple regression is used to answer the research questions. The issue of estimation of standard error has to be 
addressed first because the sampling method in TIMSS and PIRLS is stratified cluster sampling instead of 
simple random sampling. To correctly estimate sampling error, the Jackknife repeated replication methodology 
was applied. To correctly estimate measurement error, the plausible-value method was applied. The IDE 
Analyzer developed by the lEA Data Processing and Research Center (2013) was used with the statistical 
software SPSS to implement the two above methods in regression analysis. 

To test the significance of the differences between group means of science achievement. Model 0 was analyzed. 
Model 0: 

SA = Cg + ■ WAB + ■ WNO 

WAB stands for “using worksheets as a basis” and WNO stands for “not using worksheets.” The reference 
group is students taught by teachers “using worksheets as supplements (WSP).” The value of cwab is the 
difference in average science achievements for students taught by teachers WAB and WSP. The value of Cwno is 
the difference in average science achievements for students taught by teachers WNO and WSP. 

To examine the effect of controlling the four teacher and school variables (EAS, SOS, CTS, and lES), Model 1 
and Model 2 were compared. 

Model 1: 

SA = Co +c^g -WB 

WB stands for “using worksheets as a basis.” The value of cwb is the difference in average science achievements 
for students taught by teachers using worksheets as a basis and not as a basis. 

Model 2: 

S A = Cq -I- • WB + ■ EAS + ■ SOS + ■ CTS + ■ lES 

To test the significance of the interaction between WB and RA in SA, Model 3 was analyzed. In Model 3, RA is 
centered on class mean because the relationship between RA and SA on an individual level is concerned and so 
RA should not include the effect of class -level and above-class-level variables (Hoffman & Gavin, 1998). 

Model 3: 

S A = Cq -t- C^g ' WB -t- C ' EAS “t“ C * SOS C gjg * CTS C fg^ ' lES 
+ Cg^ ' (RA — RAclass^ + ^WB RA ' WB ■ (RA — RAclass^ 

To test the significance of the interaction between WB and LR on S A, Model 4 was analyzed. 

Model 4: 

S A = Cq -f ■ WB -t- C g^^ ' EAS “t“ C ^Q^ ' SOS “t“ Cgj^ ' CTS “t“ C ^gg ' lES 
+ Cg^ ■ (RA — RAclaSs) + C^yg ' W B ' (RA ~ RA class) 

~^^LR ' ^WR LR ' WB ■ LR 
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Results 

Association between Worksheet Usage and Science Achievement 

As shown in Table 1, on average across countries, 42% of students used worksheets as a basis, 56% of students 
used worksheets as supplements, and only 2% of students did not use worksheets. The percentage of students 
not using worksheets is larger than 5% only in five countries, Australia, Finland, Iran, Malta, and Sweden, and 
is zero in five countries, Croatia, Georgia, Norway, Singapore, and Taiwan. 

Table 1 shows that the difference in average science achievements for students taught by teachers using 
worksheets as a basis and as supplements is significant in eight out of the 32 countries. The average science 
achievement of students taught by teachers using worksheets as a basis is higher in only two of the eight 
countries, the Czech Republic and Saudi Arabia. The international average difference is significantly less than 
zero (-4.2). 

Association between Worksheet Usage and Science Achievement after Controlling Teacher and School 
Variables 

Table 2 provides the information about the difference in average science achievements between students taught 
by teachers using worksheets as a basis and not as a basis (as seen in the column of WB in Model 1. i.e., the 
coefficient of WB). As shown in the coefficient of WB in Model 2 (Table 2), after controlling the variables of 
EAS, SOS, CTS, and lES, the difference in average science achievements for students taught by teachers using 
worksheets as a basis and those taught by teachers not using them as a basis is significantly different from zero 
in six countries. In five out of the six countries, Italy, Northern Ireland, Qatar, Saudi Arabia, and the United 
Arab Emirates, the difference is significant in Model 1, too. The sixth country is Germany, in which the 
difference is insignificant in Model 1 . After controlling the four teacher and school variables, the coefficient of 
WB is insignificant in the Czech Republic, Malta, and Singapore while the coefficient is significant before 
controlling the four variables (Model 1) in these three countries. 


Interaction between Worksheets as a Basis and Reading Achievement 

The coefficient of WB*RA in Model 3 (Table 2) show there is no statistically significant interaction between 
worksheets as a basis and reading achievement in the prediction of science achievement in all countries. 


Interaction between Worksheets as a Basis and Class’s Lack of Readiness 

According to the information provided by Table 2 in the column WB*LR, the interaction between WB and LR 
in students’ science achievement is significant in eight out of the 32 countries. In the eight countries, the 
coefficient of the interaction term is positive in six countries and negative in two countries. 

The coefficient is positive for students in Australia, Einland, Morocco, Norway, Qatar, and the United Arab 
Emirates. The positive value of the coefficient of WB*LR means that the effect of WB on science achievement 
for students in classes lacking readiness is higher than it for those in classes not lacking readiness. Eor example, 
in Einland, after controlling EAS, SOS, CTS, EIS, and RA, the association of WB and SA for students in classes 
not lacking readiness (LR=0) is -1.4 and is not significantly different from zero; the association for students in 
classes lacking readiness (LR=1) is 8.6 (=-1.4 h- 10.0), which is significantly different from zero. In other words, 
in Einland, worksheets are more effective for students in classes lacking readiness to learn science. In Qatar and 
the United Arab Emirates, the situation is slightly different. In both countries in which the association for 
students in classes not lacking readiness (LR=0) is negative (-64.3 and -33.2), the association for students in 
classes lacking readiness (LR=1) is still negative (-38.1 and -10.5). In spite of this, the effect size becomes 
smaller, which means the negative association of worksheets is reduced when they are applied to classes lacking 
readiness. 

On the contrary, in Italy and Malta, the coefficient is negative. Worksheets in both countries are less effective 
for students in classes lacking readiness to learn science. After controlling EAS, SOS, CTS, EIS, and RA, the 
association of WB and SA for students in classes not lacking readiness (LR=0) is -11.7 for Italy and 5.1 for 
Malta; the association for students in classes lacking readiness (LR=1) is -31.5 for Italy and 1.2 for Malta. 
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Table 1: Worksheet usage and average science achievement (results from Model 0) 


Country 

Worksheet As a Basis 

Worksheet as Supplements 

Worksheet Not Used 

SAwab 
— SAwsp 

SAwno 
— SAwsp 

Percent of 
Students 

Average SA 

Percent of 
Students 

Average SA 

Percent of 
Students 

Average SA 

Australia 

15.5 (3.2) 

504.8 (8.7) 

75.9 (3.7) 

518.5(4.0) 

8.6 (2.4) 

513.2 (9.4) 

-13.8 (9.5) 

-5.3 (10.5) 

Austria 

33.3 (2.9) 

530.2 (3.8) 

66.5 (2.9) 

526.5 (3.0) 

0.2 (0.2) 

532.6 (5.8) 

3.7 (4.4) 

6.1 (6.9) 

Azerbaijan 

33.9 (3.8) 

448.2 (9.0) 

65.4 (3.9) 

426.6 (7.8) 

0.7 (0.5) 

410.3 (11.4) 

21.6(11.5) 

-16.2(14.2) 

Croatia 

28.7 (3.4) 

511.7 (3.0) 

71.3 (3.4) 

513.0(2.2) 

0.0 (0.0) 

-- 

-1.2 (3.6) 

-- 

Czech Rep. 

44.9 (3.8) 

538.3 (3.7) 

51.9 (3.8) 

527.9 (3.1) 

3.2 (1.6) 

539.9 (10.1) 

10.4 (4.9)* 

12.0(10.6) 

Finland 

39.6 (3.0) 

565.4 (3.3) 

54.2 (3.3) 

570.3 (2.6) 

6.2 (1.7) 

565.3 (12.4) 

-4.9 (3.9) 

-5.0(13.1) 

Georgia 

53.7 (4.1) 

449.4 (6.0) 

46.3 (4.1) 

451.8 (5.6) 

0.0 (0.0) 

-- 

-2.4 (9.0) 

-- 

Germany 

58.4 (3.5) 

523.3 (3.1) 

41.4 (3.5) 

529.3 (3.4) 

0.1 (0.1) 

519.4(11.7) 

-6.0 (4.7) 

-9.9 (12.4) 

Hong Kong 

44.3 (4.8) 

543.7 (4.3) 

55.0 (4.7) 

532.1 (4.1) 

0.7 (0.7) 

548.1 (6.9) 

11.6 (6.1) 

16.0 (7.2)* 

Hungary 

70.3 (3.3) 

528.9 (4.8) 

27.7 (3.4) 

531.2(7.9) 

2.0 (0.9) 

537.9 (20.7) 

-2.3 (10.3) 

6.6 (23.2) 

Iran 

14.9 (3.3) 

438.1 (11.5) 

78.8 (3.5) 

454.1 (4.5) 

6.3 (1.5) 

398.9 (20.6) 

-16.0(13.5) 

-55.2(21.4)* 

Ireland 

11.7 (2.3) 

503.9 (8.9) 

85.5 (2.6) 

514.1 (3.4) 

2.9 (1.2) 

521.4(11.5) 

-10.2 (9.8) 

7.3 (11.2) 

Italy 

23.3 (3.3) 

505.7(7.1) 

76.0 (3.2) 

528.1 (2.6) 

0.7 (0.7) 

458.4 (7.0) 

-22.4 (7.5)* 

-69.7 (0.0) 

Lithuania 

69.7 (3.6) 

509.8 (2.8) 

30.1 (3.5) 

510.0(5.1) 

0.2 (0.2) 

462.2 (8.4) 

-0.2 (6.0) 

-47.8(10.5)* 

Malta 

33.7 (0.1) 

437.3 (2.6) 

58.1 (0.1) 

444.5 (2.1) 

8.2 (0.1) 

438.2 (3.8) 

-7.3 (2.7)* 

-6.3 (4.2) 

Morocco 

67.9 (3.3) 

240.8 (5.6) 

28.0 (3.3) 

243.0 (10.3) 

4.2 (1.5) 

275.6 (35.9) 

-2.2 (12.7) 

32.7 (38.5) 

North. Ireland 

16.3 (3.0) 

498.9 (6.7) 

82.2 (3.2) 

516.7 (3.3) 

1.5 (1.1) 

489.8 (22.6) 

-17.8 (7.5)* 

-26.8(23.4) 

Norway 

39.2 (5.2) 

489.4 (4.2) 

60.8 (5.2) 

488.6 (2.4) 

0.0 (0.0) 

-- 

0.8 (4.6) 

-- 

Oman 

45.5 (3.1) 

361.9 (5.6) 

54.3 (3.1) 

376.1 (5.7) 

0.2 (0.1) 

353.9(103.6) 

-14.2 (8.1) 

-22.2(103.4) 

Poland 

57.8 (3.8) 

499.0 (3.1) 

41.9 (3.9) 

500.3 (3.0) 

0.4 (0.4) 

556.4 (7.3) 

-1.3 (4.2) 

56.0 (0.0) 

Portugal 

34.5 (4.0) 

523.8 (4.8) 

63.6(4.1) 

516.2(5.4) 

1.9 (1.5) 

488.3 (7.0) 

7.6 (7.0) 

-27.8 (8.6)* 

Qatar 

57.0 (2.9) 

359.2 (7.4) 

41.5 (3.1) 

419.4 (10.7) 

1.5 (0.9) 

492.3 (20.7) 

-60.1(14.1)* 

72.9 (25.9)* 

Romania 

35.8 (4.1) 

499.0 (9.0) 

64.0 (4.2) 

501.2 (8.5) 

0.1 (0.1) 

438.9 (17.3) 

-2.2(12.5) 

-62.3(19.1)* 

Russian Fed. 

47.7 (4.2) 

551.5 (4.7) 

50.8 (4.1) 

545.8 (4.0) 

1.4 (0.8) 

548.7 (66.6) 

5.8 (5.8) 

3.0 (66.8) 

Saudi Arabia 

51.6(4.0) 

441.7(7.3) 

46.6 (3.9) 

402.4 (9.0) 

1.8 (0.9) 

441.4(46.5) 

39.3(12.3)* 

39.0 (48.2) 

Singapore 

68.6 (2.6) 

574.6 (4.3) 

31.4 (2.6) 

589.4 (5.6) 

0.0 (0.0) 

-- 

-14.8 (7.3)* 

-- 

Slovak Rep. 

38.5 (3.0) 

530.3 (5.3) 

58.9 (3.2) 

527.8 (3.8) 

2.6 (1.3) 

474.1 (22.7) 

2.5 (5.7) 

-53.7(23.4)* 

Slovenia 

50.3 (3.8) 

512.5 (2.7) 

48.2 (3.9) 

520.3 (3.3) 

1.5 (0.8) 

504.8 (21.0) 

-7.8 (4.6) 

-15.5(20.6) 

Spain 

34.0 (3.7) 

507.9 (5.2) 

63.9 (3.7) 

498.7 (3.6) 

2.0 (0.9) 

495.3 (17.7) 

9.2 (6.5) 

-3.5 (18.1) 

Sweden 

18.5 (3.8) 

527.5 (8.0) 

67.5 (4.7) 

531.4(4.0) 

14.0 (3.6) 

539.9 (4.6) 

-3.9 (8.2) 

8.6 (6.1) 

Taiwan 

44.2(4.1) 

549.0 (2.6) 

55.8 (4.1) 

547.6 (3.0) 

0.0 (0.0) 

-- 

1.4 (4.3) 

-- 

UAE 

53.5 (2.4) 

406.5 (3.9) 

45.9 (2.4) 

442.9 (5.2) 

0.6 (0.2) 

440.2 (42.5) 

-36.4 (7.7)* 

-2.7 (42.9) 

Int. Average 

41.8 (0.6) 

487.9(1.0) 

55.9 (0.6) 

492.1 (0.9) 

2.3 (0.2) 

480.9 (5.8) 

-4.2 (1.4)* 

-6.3 (6.0) 


Note: SAwab =Average science achievement of students using worksheets as a basis; SAwsp = Average science achievement of students 
using worksheets as supplements; SAwno = Average science achievement of students not using worksheets; UAE = United Arab Emirates 
(): Standard errors appear in parentheses. */?< .05 
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Table 2: Coefficients for multiple regression analysis 


n 

o 

g 

S 

o 

Worksheet 
as a Basis 

Reading 

WB*RA 

Lack of 
Readiness 

WB*LR 


Controlled Variable 



&• 

(WB) 

Ach. (RA) 

(LR) 

EAS 

SOS 

CTS 

lES 


Ml 

-13.2 (9.4) 









c 

fZ) 

M2 

-8.2 (9.6) 





3.2 (2.1) 

7.1(2.2)* 

0.7 (1.5) 

1.8 (1.9) 


M3 

-5.7 (9.6) 

0.76 (0.02)* 

0.03 (0.05) 



3.1 (2.1) 

7.1(2.2)* 

1.0 (1.4) 

1.7 (1.9) 


M4 

-14.0(12.5) 

0.76 (0.02)* 

0.03 (0.04) 

-17.2(3.0)* 

35.4(15.6)* 

1.0 (2.1) 

5.9(2.2)* 

2.0 (1.3) 

0.1 (1.9) 


Ml 

3.7 (4.4) 









> 

c 

M2 

-0.1 (4.2) 





4.8 (1.4)* 

4.4(1.5)* 

-2.4(1.2)* 

-1.7 (1.3) 

S' 

M3 

-0.6 (4.2) 

0.85 (0.01)* 

-0.04(0.03) 



4.8 (1.4)* 

4.4(1. 6)* 

-2.2 (1.2) 

-1.6 (1.2) 


M4 

-0.8 (4.2) 

0.84 (0.01)* 

-0.04(0.03) 

-3.7 (2.4) 

-2.7 (3.8) 

4.3 (1.4)* 

3.8(1.6)* 

-2.5(1.2)* 

-2.0 (1.3) 

> 

Ml 

21.8(11.5) 









N 

CK 

M2 

18.5(11.5) 





4.7 (2.6) 

0.0 (3.1) 

2.7 (4.9) 

1.9 (3.4) 

£5. 

M3 

18.4(11.4) 

0.54 (0.04)* 

-0.04(0.07) 



4.7 (2.6) 

0.0 (3.1) 

2.6 (5.0) 

1.7 (3.4) 

3 

M4 

21.0(12.7) 

0.52 (0.04)* 

-0.03(0.08) 

-7.8 (6.3) 

5.0 (9.6) 

3.4 (3.0) 

-0.7 (3.2) 

5.2 (4.7) 

-0.3 (3.4) 


Ml 

-1.2 (3.6) 









Q 

o 

M2 

-0.7 (3.6) 





3.5 (1.4)* 

-1.2 (1.2) 

-1.6 (1.1) 

1.3 (1.2) 

3 

S' 

M3 

-1.5 (3.7) 

0.77 (0.02)* 

-0.01(0.02) 



0.5 (0.9) 

-0.8 (1.2) 

-1.5 (1.0) 

1.2 (1.2) 


M4 

0.8 (4.2) 

0.77 (0.02)* 

-0.01(0.02) 

-2.2 (2.4) 

5.6 (3.7) 

0.6 (0.9) 

-0.8 (1.2) 

-1.5 (1.0) 

1.4 (1.1) 


Ml 

9.7 (4.8)* 









_ 

o o 

c 

M2 

9.0 (4.9) 





0.6 (0.9) 

-1.8 (1.1) 

0.7 (1.4) 

1.4 (1.5) 

s s- 

o 

M3 

9.9 (4.8)* 

0.88 (0.03)* 

0.01 (0.03) 



2.3 (1.4) 

-1.5 (1.2) 

0.6 (1.4) 

1.4 (1.5) 


M4 

8.7 (4.9) 

0.88 (0.03)* 

0.01 (0.03) 

-6.7 (3.8) 

1.3 (5.4) 

1.9 (1.4) 

-1.7 (1.2) 

0.3 (1.4) 

1.5 (1.3) 


Ml 

-4.4 (3.9) 










M2 

-4.4 (3.7) 





2.2 (1.5) 

5.0(1. 0)* 

-0.5 (1.1) 

0.7 (1.3) 

S' 

3 

Cl 

M3 

-4.7 (3.5) 

0.73 (0.02)* 

0.00 (0.03) 



1.0 (1.4) 

5.2(1. 0)* 

-0.2 (1.1) 

0.2 (1.4) 


M4 

-1.4 (3.8) 

0.73 (0.02)* 

0.00 (0.03) 

-7.7 (2.2)* 

10.0 (3.6)* 

0.9 (1.3) 

4.2(1.!)* 

0.0 (1.0) 

-0.6 (1.4) 


Ml 

-2.4 (9.0) 









C 

CK 

O 

M2 

-4.2 (9.1) 





1.2 (1.4) 

1.8 (2.8) 

-0.9 (2.7) 

-0.7 (2.2) 

S' 

M3 

-4.5 (9.1) 

0.82 (0.02)* 

-0.03(0.02) 



5.1 (2.2)* 

2.0 (2.8) 

-0.8 (2.6) 

-0.5 (2.2) 


M4 

-9.7 (9.2) 

0.82 (0.02)* 

-0.02(0.02) 

2.2 (4.2) 

-7.6 (5.7) 

4.9 (2.2)* 

2.0 (2.9) 

-0.7 (2.6) 

-0.4 (2.1) 

o 

Of 

3 

Ml 

-5.9 (4.7) 









M2 

-8.8 (3.4)* 





5.1 (2.2)* 

2.3 (1.4) 

-0.9 (1.4) 

-4.3(1.3)* 

3 

3 

3 

<< 

M3 

-8.4 (3.3)* 

0.78 (0.04)* 

0.03 (0.03) 



10.4(1.8)* 

2.6 (1.3) 

-0.8 (1.4) 

-4.6(1.3)* 


M4 

-7.2 (3.4)* 

0.78 (0.04)* 

0.03 (0.03) 

-4.4 (3.3) 

1.0 (4.5) 

9.8 (1.8)* 

3.0(1.3)* 

-1.1 (1.4) 

-4.8(1.3)* 

:c 

Ml 

11.4 (6.1) 









o 

3 

CfQ 

M2 

10.3 (6.0) 





10.7(1.8)* 

1.4 (1.9) 

-3.3(1.4)* 

-1.1 (1.6) 

3 

M3 

9.8 (6.0) 

0.72 (0.03)* 

0.01 (0.04) 



-0.1 (1.6) 

1.5 (2.0) 

-3.5(1.4)* 

-1.1 (1.6) 

CfQ 

M4 

7.5 (6.1) 

0.72 (0.03)* 

0.01 (0.04) 

-3.5 (3.7) 

-6.8 (4.1) 

-0.4 (1.5) 

0.7 (2.0) 

-3.8(1.3)* 

-1.6 (1.6) 


Ml 

-2.8 (9.8) 









X 

3 

3 

M2 

-8.7 (8.4) 





0.0 (1.6) 

4.6 (2.4) 

-4.7 (1.8)* 

-2.1 (1.9) 

CfQ 

n 

M3 

-10.1 (8.5) 

0.82 (0.02)* 

0.02 (0.03) 



11.6(2.0)* 

5.0(2.4)* 

-5.0 (1.9)* 

-1.9 (1.9) 


M4 

-10.8 (8.4) 

0.82 (0.02)* 

0.02 (0.03) 

-0.5 (5.9) 

-5.1 (7.2) 

11.3(2.0)* 

4.1 (2.4) 

-5.0 (1.9)* 

-2.7 (1.9) 


Ml 

-11.9(13.5) 









- 

M2 

-13.0(14.3) 





11.9(2.0)* 

4.1 (2.4) 

-2.5 (2.8) 

3.9 (2.5) 

3 

M3 

-13.4(14.3) 

0.87 (0.02)* 

-0.02(0.04) 



7.4 (2.5)* 

3.9 (2.4) 

-2.5 (2.8) 

3.9 (2.5) 


M4 

-17.4(13.8) 

0.87 (0.02)* 

-0.02(0.04) 

-8.9 (5.7) 

-15.2(11.9) 

7.1 (2.5)* 

1.9 (2.5) 

-4.4 (3.0) 

4.6 (2.4) 


Note: EAS = Emphasis on Academic Success; SOS = Safety and Orderliness of School; CTS = Confidence in Teaching Science; lES = 
Instructional Engagement of Students. 

(): Standard errors appear in parentheses. */?< .05 
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Table 2 {cont.)\ Coefficients for multiple regression analysis 


o 

o 

c 

S 

o 

Worksheet 
as a Basis 

Reading 

WB*RA 

Lack of 
Readiness 

WB*LR 


Controlled Variable 


'5 

&• 

2. 

(WB) 

Ach. (RA) 

(LR) 

EAS 

SOS 

CTS 

lES 


Ml 

-10.4 (9.8) 










M2 

-4.8 (9.5) 





7.3 (2.5)* 

5.9(1.8)* 

3.4(1. 6)* 

-4.7(1. 7)* 

a. 

M3 

-4.3 (9.8) 

0.78 (0.02)* 

0.04 (0.04) 



5.9 (1.5)* 

6.1(1.7)* 

3.1 (1.6) 

-4.9(1. 6)* 


M4 

-2.6 (9.6) 

0.78 (0.02)* 

0.04 (0.04) 

-4.7 (2.5) 

6.8 (8.0) 

5.3 (1.5)* 

5.5(1.7)* 

2.7 (1.6) 

-4.7(1. 7)* 


Ml 

-21.8(7.5)* 










M2 

-18.9(8.0)* 





5.7 (1.5)* 

3.0 (2.1) 

-0.2 (1.9) 

-1.8(1.4) 


M3 

-19.0(7.9)* 

0.78 (0.02)* 

-0.02(0.04) 



0.0 (1.7) 

3.0 (2.1) 

-0.3 (1.9) 

-1.5 (1.4) 


M4 

-11.7 (7.8) 

0.78 (0.02)* 

-0.01(0.04) 

10.4 (4.2)* 

-19.8(6.5)* 

1.1 (1.7) 

3.7 (2.1) 

-0.7 (1.9) 

-2.8(1.3)* 


Ml 

0.2 (5.9) 









B- 

c 

M2 

-2.7 (5.8) 





0.2 (1.7) 

1.2 (1.6) 

-1.3 (1.7) 

0.1 (1.6) 


M3 

-3.1 (5.7) 

0.75 (0.03)* 

0.01 (0.04) 



5.4 (2.0)* 

1.1 (1.6) 

-1.5 (1.7) 

0.2 (1.6) 


M4 

-4.0 (6.1) 

0.75 (0.03)* 

0.01 (0.04) 

-7.1 (5.2) 

1.2 (5.5) 

4.7 (2.0)* 

1.1 (1.6) 

-1.9 (1.7) 

-0.1 (1.6) 


Ml 

-6.5 (2.6)* 









S 

M2 

3.6 (2.6) 





5.8 (1.9)* 

4.4(0.7)* 

2.8(0.6)* 

-6. 1(0.8)* 


M3 

4.2 (1.6)* 

0.74 (0.02)* 

0.00 (0.04) 



5.1 (0.5)* 

4.2(0.5)* 

2.8(0.4)* 

-6. 1(0.5)* 


M4 

5.1 (1.5)* 

0.74 (0.02)* 

0.00 (0.04) 

0.5 (0.9) 

-3.9 (1.5)* 

5.1 (0.5)* 

4. 1(0.6)* 

2.9(0.4)* 

-6.0(0.5)* 


Ml 

-6.5 (12.4) 









2 

o 

M2 

-9.5 (12.8) 





4.8 (0.6)* 

6.3(3.0)* 

1.2 (4.1) 

3.5 (3.1) 

o 

O 

O 

o 

M3 

-7.4 (12.8) 

0.66 (0.05)* 

0.01 (0.06) 



6.7 (4.3) 

6.0(3.0)* 

1.6 (4.1) 

3.4 (3.0) 


M4 

-39.7(21.5) 

0.66 (0.05)* 

0.01 (0.06) 

-51.2(23.1)* 

57.2(25.1)* 

7.7 (4.3) 

5.7(2.8)* 

1.0 (4.1) 

1.3 (2.9) 


Ml 

-17.3(7.5)* 









^ §. 

M2 

-18.6(6.9)* 





7.1 (4.3) 

5.8(1.8)* 

-0.1 (1.5) 

-1.7 (1.8) 

^ B 

5 2 

6 3 

M3 

-19.4(6.9)* 

0.69 (0.01)* 

-0.04(0.06) 



3.3 (1.7)* 

6.0(1.8)* 

-0.1 (1.5) 

-2.0 (1.8) 


M4 

-22.5(6.9)* 

0.70 (0.01)* 

-0.04(0.05) 

-6.8 (2.9)* 

-10.7 (5.8) 

2.2 (1.6) 

4.7(1.7)* 

0.2 (1.4) 

-1.9 (1.9) 


Ml 

0.8 (4.6) 









Z 

o 

M2 

1.0 (4.0) 





3.2 (1.7) 

3.0 (1.6) 

-2.5 (1.4) 

1.4 (1.5) 


M3 

0.7 (4.1) 

0.78 (0.03)* 

0.02 (0.04) 



5.2 (1.5)* 

3.1(1.6)* 

-2.4 (1.3) 

1.0 (1.4) 


M4 

4.7 (4.8) 

0.78 (0.03)* 

0.02 (0.04) 

-9.8 (2.3)* 

11.2(4.5)* 

4.8 (1.4)* 

3.2(1.5)* 

-2.8 (1.4) 

0.6 (1.4) 


Ml 

-14.1 (8.1) 









O 

3 

M2 

-12.0(7.1) 





5.1 (1.5)* 

5.8(2.8)* 

-3.6 (2.7) 

2.2 (2.5) 

3 

M3 

-11.4(7.2) 

0.92 (0.02)* 

0.01 (0.02) 



7.2 (2.3)* 

5.5(2.8)* 

-3.6 (2.7) 

2.3 (2.5) 


M4 

-14.4(7.1)* 

0.92 (0.02)* 

0.01 (0.02) 

-2.3 (7.1) 

3.6 (8.9) 

6.8 (2.4)* 

5.0 (2.9) 

-2.5 (2.7) 

2.3 (2.6) 


Ml 

-1.8 (4.1) 









Poland 

M2 

M3 

-4.4 (3.9) 
-5.3 (3.9) 

0.84 (0.02)* 

0.01 (0.03) 



7.2 (2.4)* 
3.6 (1.5)* 

3.6(1.6)* 

-3.2(1.6)* 

1.7 (1.5) 
1.6 (1.5) 

-1.3 (1.1) 
-1.4 (1.1) 


M4 

-5.3 (3.9) 

0.83 (0.02)* 

0.01 (0.03) 

-1.8 (2.7) 

-2.6 (4.2) 

3.6 (1.5)* 

-4.0(1.5)* 

1.6 (1.5) 

-1.5 (1.2) 


Ml 

8.4 (7.0) 









o 

M2 

8.3 (7.8) 





3.8 (1.4)* 

0.6 (2.7) 

-0.8 (2.1) 

-1.6 (1.5) 

3 

Cfci 

M3 

7.5 (7.9) 

0.77 (0.02)* 

-0.05(0.03) 



9.6 (2.2)* 

0.9 (2.7) 

-0.7 (2.1) 

-1.5 (1.5) 


M4 

7.6 (7.5) 

0.77 (0.02)* 

-0.05(0.03) 

-0.4 (6.2) 

1.0 (7.2) 

9.6 (2.5)* 

0.9 (2.7) 

-0.8 (1.9) 

-1.5 (1.5) 


Ml 

-62.7(13.4)* 









o 

M2 

-61.4(14.0)* 





9.4 (2.2)* 

4.7 (3.1) 

0.5 (4.2) 

-3.4 (4.0) 


M3 

-61.4(14.0)* 

0.83 (0.03)* 

0.05 (0.04) 



5.5 (4.4) 

4.9 (3.1) 

0.8 (4.2) 

-3.3 (4.0) 


M4 

-64.3(13.2)* 

0.83 (0.03)* 

0.05 (0.04) 

-38.1(8.0)* 

26.2(9.8)* 

3.3 (4.1) 

3.7 (3.5) 

0.1 (3.6) 

-4.0 (3.7) 


Note: EAS = Emphasis on Academic Success; SOS = Safety and Orderliness of School; CTS = Confidence in Teaching Science; lES = 
Instructional Engagement of Students. 

(): Standard errors appear in parentheses. */?< .05 
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Table 2 (cont.): Coefficients for multiple regression analysis 


o 

o 

2 

Worksheet 



Lack of 



Controlled Variable 


c 

o 

as a Basis 

Reading 

WB*RA 

Readiness 

WB*LR 





'5 


(WB) 

Ach. (RA) 


(LR) 


EAS 

SOS 

CTS 

lES 


Ml 

-2.0 (12.5) 










o 

B 

M2 

-0.8 (12.5) 






5.9 (4.3) 

-2.6 (3.9) 

-11.4(5.0)* 

-0.8 (2.8) 

S’ 

M3 

-2.0 (12.5) 

0.85 

(0.03)* 

-0.01(0.03) 



10.2(3.1)* 

-2.8 (3.9) 

-11.8(5.0)* 

-0.5 (2.9) 

M4 

-3.7(11.6) 

0.85 

(0.03)* 

0.00 (0.03) 

-25.4(8.5)* 

5.9 (9.9) 

7.7 (2.9)* 

-5.3 (4.0) 

-9.0 (4.8) 

-0.8 (3.0) 


Ml 

5.7 (5.7) 










g. ^ 

^ C 

2 t/3 

M2 

5.1 (5.6) 






9.8 (3.1)* 

-0.3 (2.3) 

0.0 (3.3) 

2.3 (1.9) 

s 

o' 5 

M3 

4.5 (5.6) 

0.71 

(0.03)* 

-0.01(0.03) 



3.0 (2.0) 

-0.3 (2.4) 

-0.2 (3.3) 

2.3 (1.9) 

3 

M4 

6.3 (6.7) 

0.71 

(0.03)* 

-0.01(0.03) 

-2.5 (4.5) 

4.6 (6.1) 

2.5 (2.3) 

-0.3 (2.5) 

-0.1 (3.3) 

2.4 (1.9) 


Ml 

37.9(12.1)* 










> 00 

M2 

23.8(12.1)* 






2.8 (2.0) 

-1.6 (3.7) 

4.1 (4.2) 

0.5 (2.7) 

PD -• 

M3 

23.2 (12.0) 

0.82 

(0.02)* 

0.00 (0.04) 



7.6 (3.0)* 

-1.9 (3.6) 

4.5 (4.3) 

0.8 (2.7) 


M4 

32.9(11.6)* 

0.82 

(0.02)* 

-0.01(0.04) 

8.0 (8.2) 

-14.4(10.4) 

7.3 (3.3)* 

-2.8 (3.6) 

3.5 (4.4) 

1.3 (2.7) 


Ml 

-14.8 (7.3)* 










5' 

(TCI 

M2 

-12.1 (7.0) 






8.1 (3.0)* 

3.6 (2.4) 

-2.2 (1.8) 

-2.1 (1.5) 

O 

M3 

-13.1 (7.0) 

0.81 

(0.03)* 

-0.01(0.04) 



6.7 (2.4)* 

3.3 (2.4) 

-2.2 (1.8) 

-1.9 (1.5) 

CD 

M4 

-11.3 (6.3) 

0.80 

(0.03)* 

0.00 (0.03) 

-26.7(7.1)* 

0.7 (7.8) 

4.8 (2.2)* 

1.0 (2.2) 

-3.1 (1.7) 

-3.4(1.5)* 


Ml 

4.8 (5.8) 










^ a 

c S 

M2 

0.2 (5.8) 






6.6 (2.4)* 

-0.4 (2.8) 

-1.3 (1.6) 

-2.3 (1.9) 

1 ^ 

M3 

1.0 (5.8) 

0.85 

(0.02)* 

-0.04(0.03) 



9.8 (3.0)* 

-0.4 (2.8) 

-1.1 (1.6) 

-2.4 (1.9) 


M4 

-2.4 (8.1) 

0.85 

(0.02)* 

-0.04(0.03) 

-7.6 (3.4)* 

-1.0 (5.7) 

7.8 (3.1)* 

-0.5 (3.0) 

-1.0 (1.6) 

-2.9 (2.0) 


Ml 

-7.3 (4.7) 










'o 

< 

M2 

-7.5 (4.9) 






9.8 (3.0)* 

1.2 (1.3) 

0.1 (1.0) 

-2.1 (1.3) 

CD 

3 

S' 

M3 

-7.5 (4.8) 

0.83 

(0.02)* 

0.03 (0.03) 



3.2 (1.4)* 

1.1 (1.3) 

0.2 (1.0) 

-2.0 (1.2) 

M4 

-8.2 (4.7) 

0.83 

(0.02)* 

0.03 (0.03) 

-12.3(4.3)* 

7.6 (4.8) 

2.1 (1.5) 

0.7 (1.2) 

0.4 (1.0) 

-2.2 (1.2) 


Ml 

9.3 (6.5) 










00 

T3 

M2 

4.8 (6.6) 






3.1 (1.4)* 

4.6(1.7)* 

1.0 (1.8) 

-1.0 (1.6) 

5' 

M3 

5.5 (6.5) 

0.75 

(0.03)* 

0.04 (0.04) 



3.0 (1.8) 

5.0(1.6)* 

0.9 (1.8) 

-1.2 (1.6) 


M4 

5.3 (7.1) 

0.75 

(0.02)* 

0.04 (0.04) 

-6.4 (3.3) 

2.4 (5.1) 

2.6 (1.8) 

4.3(1.6)* 

1.2 (1.9) 

-1.4 (1.7) 


Ml 

-5.4 (7.9) 










00 

CD 

M2 

-11.6(7.3) 






3.0 (1.8) 

7.6(1.6)* 

-3.0 (1.4)* 

2.8 (1.4)* 

a. 

CD 

3 

M3 

-11.8(7.3) 

0.84 

(0.03)* 

0.05 (0.04) 



3.7 (1.6)* 

7.3(1.6)* 

-3.1 (1.5)* 

2.8 (1.4)* 


M4 

-20.6 (10.7) 

0.84 

(0.03)* 

0.05 (0.04) 

2.5 (2.7) 

-7.7 (8.2) 

4.3 (1.5)* 

7.1(1.5)* 

-2.6 (1.4) 

3.1 (1.4)* 


Ml 

1.4 (4.3) 










H 

M2 

-0.5 (4.2) 






3.1 (1.5)* 

-0.2 (1.4) 

1.1 (1.4) 

-0.2 (1.1) 

i 

3 

M3 

-0.6 (4.2) 

0.82 

(0.02)* 

-0.01(0.02) 



3.7 (1.4)* 

-0.2 (1.4) 

1.1 (1.4) 

-0.3 (1.1) 


M4 

-1.0 (4.4) 

0.82 

(0.02)* 

-0.01(0.02) 

-4.2 (3.3) 

2.9 (3.9) 

3.3 (1.5)* 

-0.3 (1.4) 

1.1 (1.4) 

-0.4 (1.1) 


Ml 

-36.4 (7.7)* 










a 

M2 

-34.1 (7.6)* 






7.7 (2.0)* 

2.2 (2.4) 

2.1 (3.1) 

-0.5 (1.7) 


M3 

-33.4 (7.6)* 

0.84 

(0.02)* 

0.02 (0.02) 



7.8 (2.0)* 

2.1 (2.4) 

2.4 (3.1) 

-0.7 (1.7) 


M4 

-33.2 (7.3)* 

0.84 

(0.02)* 

0.02 (0.02) 

-30.7(4.9)* 

22.7(7.0)* 

5.3 (1.9)* 

0.4 (2.2) 

1.8 (3.0) 

-1.8 (1.7) 


Note: EAS = Emphasis on Academic Success; SOS = Safety and Orderliness of School; CTS = Confidence in Teaching Science; lES = 
Instructional Engagement of Students; UAE = United Arab Emirates. 

(): Standard errors appear in parentheses. *p< .05 


Discussion and Conclusion 

Instructional written materials play important roles as teachers’ agents in effective teaching practices. 
Workbooks and worksheets are one of the most frequently used materials (Table 1). Based on the result of this 
study, the association of worksheet usage and science achievement is found to be quite different across 
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countries. To sum up the result, there are five types of relationships among science achievement, worksheet 

usage, and other related variables (Table 3). 

• Type 1; The association between WB and SA remains the same regardless of whether or not teacher and 
school variables are controlled, and no interaction is present between WB and LR in SA. 

• Type 2; The association between WB and SA depends on whether or not teacher and school variables are 
controlled, and no interaction is present between WB and LR in SA. 

• Type 3: The association between WB and SA remains the same regardless of whether or not teacher and 
school variables are controlled, and a positive interaction is present between WB and LR in SA. 

• Type 4: The association between WB and SA remains the same regardless of whether or not teacher and 
school variables are controlled and a negative interaction is present between WB and LR in SA. 

• Type 5; The association between WB and SA depends on whether or not teacher and school variables are 
controlled, and a negative interaction is present between WB and LR in SA. 

In addition, there is no significant interaction between WB and RA in all participating countries. 


Table 3: Relationships among worksheet usage, science achievement, and other variables 


Type 

Association btw WB & SA 
after controlling variables 

WB*LR 

Country 

No 

Teacher and 
school variables 





Austria, Azerbaijan, Croatia, Czech Repub lic“, 
Georgia, Hong Kong, Hungary, Iran, Ireland, 

la 

~S 

~S 

~S 

Lithuania, Oman, Poland, Portugal, Romania, 
Russian federation, Slovak Republic, Slovenia, 
Spain, Sweden, Taiwan 

lb 

N 

N 

~S 

Northern Ireland 

Ic 

P 

P 

~S 

Saudi Arabia 

2a 

N 

~S 

~S 

Singapore 

2b 

~S 

N 

~S 

Germany 

3a 

~S 

~S 

P 

Australia, Einland, Morocco, Norway 

3b 

N 

N 

P 

Qatar, United Arab Emirates 

4 

N 

N 

N 

Italy 

5 

N 

~S 

N 

Malta 


Note: ~S = Not significantly different from zero; P = Significantly positive; N = Significantly negative. 

Although the association between WB and S A is significantly different from zero, none of the teacher and school variables 
are significantly related to SA. 


Based upon the above findings, there are four directions of further investigation to identify important features of 
designing and applying worksheets through comparisons across countries in future studies. 

Firstly, the international comparison can be made among three groups of countries to identify the related factors 
in predicting science achievement. In most countries, there is no association between WB and SA, including 
countries of type la and 3a. Only in Saudi Arabia is the association positively different from zero. In the four 
countries of type lb, 3b, and 4, the association is negatively different from zero. 

One explanation of the negative association between worksheet usage and science achievement is that teachers 
tended to use worksheets in low-achievement classes, as Reid (1984) reported. If this is true, teachers’ 
perceptions of class achievement may be the cause of the negative association. However, after the introduction 
of the variable of classes’ lack of readiness in Model 4, the association in the four countries. Northern Ireland, 
Qatar, the United Arab Emirates, and Italy, remains the same (Table 2). Consequently, there are other factors 
that have yet to be uncovered. 

The second direction of future investigation is the relationship between worksheet usage and other teacher and 
school factors. After controlling the teacher and school variables, EAS, SOS, CTS, and lES, for type 2 and 5 
countries (Singapore, Germany, and Malta), the association between WB and SA changed. Eor Singapore, the 
variable EAS is significantly related to SA in Model 2. Eor Germany, the significantly relative variables are 
EAS and lES. Eor Malta, the relative variables are EAS, SOS, CTS, and lES (Table 2). These results imply that 
these teacher and school variables, worksheet usage, and science achievement are correlated in these countries. 
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It is worth inquiring why these teacher and school variables and worksheet usage are correlated and how they 
together influence students’ science achievement. 

Thirdly, the mechanisms that make worksheets more effective for students in classes lacking readiness than 
those in classes not lacking readiness are worth looking at further. It would be easier to find out the mechanisms 
through the use of data from countries of type 3 (Australia, Finland, Morocco, Norway, Qatar, and the United 
Arab Emirates), which have positive interactions between WB and LR in Model 4 (Table 2). Data from Italy 
and Malta could also be used as contrast. To find the mechanisms, more data about worksheet design and about 
teaching and learning with worksheets should be collected. For example, strategies that students use to complete 
worksheets are important factors related to their achievement but are not well documented. 

The last direction of further inquiry is identifying the factors that result in no interaction between worksheet 
usage and reading achievement in science achievement. The result of no interaction between WB and RA may 
be caused by appropriate matching of language levels of worksheets with students’ reading abilities, but it might 
also be caused by teachers’ explanations before students starting to work on worksheets. The data collected by 
TIMSS and PIRLS 201 1 cannot help us to identify the cause. To resolve this problem, data about evaluation on 
language demand of worksheets and the teaching methods accompanying worksheets need to be collected. 


Acknowledgements or Notes 

The author wish to thank the project (NSC 101-251 l-S-003-021) funded by National Science Council of 
Taiwan. 


References 

Anderson, R. C., Hiebert, E. H., Scott, J. A., & Wilkinson, I. A. G. (1985). Becoming a nation of readers: The 
report of the Commission on Reading. Washington, DC: National Academy of Education, National Institute 
of Education, Center for the Study of Reading. 

Calderhead, W. J., Filter, K. J., & Albin, R. W. (2006). An investigation of incremental effects of interspersing 
math items on task-related behavior. Journal of Behavioral Education, 75(1), 51-65. 

Campbell, C. P. (1999). Instructional materials: Their preparation and evaluation. Journal of European 
Industrial Training, 23(2), 55-107. 

Department of Education and Science (2007). Inclusion of students with special educational needs: Post 
primary guidelines. Dublin: Stationery Office. 

Foy, P. (2013). TIMSS and PIRLS 2011 user guide for the fourth grade combined international database. 
Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston 
College and lEA. 

Hayes, D. P, Wolfer, L. T., & Wolfe, M. F. (1996). Schoolbook simplification and its relation to the decline in 
SAT- Verbal scores. American Educational Research Journal, 33(2), 489-508. 

Hoener, A., Salend, S., & Kay, S. I. (1997). Creating readable handouts, worksheets, overheads, tests, review 
materials, study guides, and homework assignments through effective typographic design. Teaching 
Exceptional Children, 29(3), 32-35. 

Hoffman, D. A., & Gavin, M. B. (1998). Centering decisions in hierarchical linear models: Implications for 
research in organizations. Journal of Management, 23, 723-744. 

lEAData Processing and Research Center (2013). IDA analyzer. Amsterdam, Netherlands: lEA. Retrieved from 
http://www.iea.nl/eula.html. 

Kisiel, J. F. (2003). Teachers, museums and worksheets: A closer look at a learning experience. Journal of 
Science Teacher Education, 74(1), 3-21. 

Krombab, A., & Harms, U. (2008). Acquiring knowledge about biodiversity in a museum - Are worksheets 
effective? Journal of Biological Education, 42(A), 157-163. 

Lesley, M., & Labbo, L. D. (2003). A pedagogy of control: Worksheets and the special need child. Language 
Arts, 80(6), 444. 

Martin, M. O., & Mullis, I. V. S. (Eds.) (2013). TIMSS and PIRLS 2011: Relationships among reading, 

mathematics, and science achievement at the fourth grade - Implications for early learning. Chestnut Hill, 
MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College and lEA. 

Martin, M. O., Mullis, I. V. S., Foy, P, & Stanco, G. M. (2012). TIMSS 2011 international results in science. 
Chestnut Hill, M.A.: TIMSS & PIRLS International Study Center. 

McDowell, E. T., Waddling, R. E. L. (1985). Improving the design of laboratory worksheets. Journal of 



106 Lee 


Chemical Education, 62(11), 1037-1038. 

Meyer, B. J. F. (2003). Text coherence and readability. Topics in Language Disorders, 23(3), 204-224. 

O'Leary, S. (2011). The inclusive classroom: Effect of a readability intervention on student engagement and on- 
task behaviour within two mixed-ability science classrooms. Science Education International, 22(2), 145- 
151. 

Reid, D. (1984). Readability and science worksheets in secondary schools. Research in Science and 
Technological Education, 2(2), 153-165. 

Rix, J. (2006). Simplified language materials: Their usage and value to teachers and support staff in mainstream 
settings. Teaching and Teacher Education, 22(8), 1145-1156. 

Rotter, K. (2006). Creating instructional materials for all pupils: Try COLA. Intervention in School and Clinic, 
41(5), 273-282. 

Sasmaz-Oren, F, & Ormanci, U. (2012). An application about pre-service teachers' development and use of 
worksheets and an evaluation of their opinions about the application. Educational Sciences: Theory and 
Practice, 12(1), 263-270. 

Ueckert, C. W., & Gess-Newsome, J. (2008). Active learning strategies. Science Teacher, 75(9), 47-52. 

Wolf, J., Stanton, M., & Gellott, L. (2010). Critical thinking in physical geography: Linking concepts of content 
and applicability. Journal of Geography, 109(2), 43-53. 



